Repository navigation
Conversation
…exit codes Wrap sample test invocations in run_with_retry (up to 3 attempts with exponential backoff and DB table reset) when transient upstream LLM errors (503 UNAVAILABLE / 429 RESOURCE_EXHAUSTED) occur, and add matching retry helpers in the Go sample test harnesses. Also replace the piped find | while loop with process substitution and capture test exit codes before cleanup so subshell failures are never masked by rm -rf.
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces transient LLM error detection and retry mechanisms with exponential backoff to the CI test runner (run_tests.sh) and Go integration tests (agent_test.go and quickstart_test.go), alongside improved error handling in the shell script. The reviewer suggested expanding the list of transient error markers in both Go test files to match the comprehensive patterns checked in the shell script (such as specific status codes and JSON-formatted errors) for more robust detection.
Contributor
Author
|
/gcbrun |
Closed
3 of 7 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wrap sample test invocations in run_with_retry (up to 3 attempts with exponential backoff and DB table reset) when transient upstream LLM errors (503 UNAVAILABLE / 429 RESOURCE_EXHAUSTED) occur, and add matching retry helpers in the Go sample test harnesses. Also replace the piped find | while loop with process substitution and capture test exit codes before cleanup so subshell failures are never masked by rm -rf.
Description
Hardens sample integration tests against transient upstream Gemini API capacity errors (
503 UNAVAILABLE/429 RESOURCE_EXHAUSTED) and fixes exit-code masking in.ci/sample_tests/run_tests.sh:run_with_retryandis_transient_llm_errorin.ci/sample_tests/run_tests.shto retry sample test runs up to 3 times with exponential backoff and a freshsetup_db_tablereset when transient 503/429 LLM errors occur.docs/en/documentation/getting-started/quickstart/go/quickstart_test.goanddocs/en/documentation/configuration/pre-post-processing/go/agent_test.gofor standalonego testruns (disabled viaSAMPLE_TEST_MAX_ATTEMPTS=1when already wrapped byrun_tests.sh).rm -rfcleanup inrun_python_test,run_js_test, andrun_go_test, and replacesfind ... | while readwithwhile read ... done < <(find ... | sort)so failures inside the loop are not lost in a pipeline subshell.PR Checklist
CONTRIBUTING.md
bug/issue
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
review
!if this involve a breaking change🛠️ Fixes #<issue_number_goes_here>